A semiconductor memory device of three-dimensional structure is under developing, which includes a charge storage layer and a semiconductor layer provided in a memory hole. The memory hole is formed in a stacked body that includes a plurality of electrode layers stacked on a conductive layer, and the charge storage layer and the semiconductor layer extend in a stacked direction of the plurality of electrode layer. In the manufacturing process of such a memory device, the memory hole is formed from a top surface to be in communication with the conductive layer.